2017-10-13

Mysql-5.5+Heartbeat-3.0.5+DRBD

环境：

CentOS 6.5

MySQL_Master

eth0 192.168.1.10

eth1 192.168.2.10

MySQL_Slave

eth0 192.168.1.11

eth1 192.168.2.11

HA 192.168.1.254

创建RAID1+0组合[存储数据四块硬盘]

1.创建两块RAID1 [4块硬盘、MySQL主从节点执行]

[root@master ~]# mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: Note: this array has metadata at the start and
 may not be suitable as a boot device. If you plan to
 store '/boot' on this device please ensure that
 your boot-loader understands md/v1.x metadata, or use
 --metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
[root@master ~]# mdadm --create /dev/md1 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: Note: this array has metadata at the start and
 may not be suitable as a boot device. If you plan to
 store '/boot' on this device please ensure that
 your boot-loader understands md/v1.x metadata, or use
 --metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.

2.利用两个 RAID1 创建 RAID0

1
2
3

[root@CentOS ~]# mdadm --create /dev/md2 --level=raid0 --raid-devices=2 /dev/md0 /dev/md1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md2 started.

3.将raid信息写入配置文件

[root@master ~]# mdadm --detail --scan > /etc/mdadm.conf 
[root@master ~]# vi /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=CentOS:0 UUID=b4c4c7b4:0f9f6e60:7eb24578:29682c96
devices /dev/sdb1 /dev/sdc1
ARRAY /dev/md1 metadata=1.2 name=CentOS:1 UUID=f5afcda6:86847677:c752fcdd:fbb91e00
devices /dev/sdd1 /dev/sde1
ARRAY /dev/md2 metadata=1.2 name=CentOS:2 UUID=00f120ec:bab2f3fe:80d88cb9:3ee4b76b
devices /dev/md0 /dev/md1

安装DRBD

1.解压并安装DRBD

[root@master Linux]# yum -y install gcc kernel-devel kernel-headers flex perl
[root@master Linux]# http://oss.linbit.com/drbd/8.4/drbd-8.4.4.tar.gz
[root@master Linux]# tar fzvx drbd-8.4.4.tar.gz
[root@master Linux]# cd drbd-8.4.4
[root@master drbd-8.4.4]# ./configure --prefix=/usr/local/drbd-8.4 --with-km
[root@master drbd-8.4.4]# make KDIR=/usr/src/kernels/2.6.32-358.el6.x86_64/
[root@master drbd-8.4.4]# make install
[root@master drbd-8.4.4# mkdir -p /usr/local/drbd-8.4/var/run/drbd
[root@master drbd-8.4.4]# cp /usr/local/drbd-8.4/etc/rc.d/init.d/drbd /etc/init.d/
[root@master drbd-8.4.4]# chkconfig --add drbd
[root@master drbd-8.4.4]# chkconfig drbd on

2.安装drbd模块

[root@master drbd-8.4.4]# cd drbd
[root@master drbd]# make clean
[root@master drbd]# make KDIR=/usr/src/kernels/2.6.32-358.el6.x86_64/
[root@master drbd]# cp drbd.ko /lib/modules/2.6.32-358.el6.x86_64/kernel/lib/
[root@master drbd]# depmod

3.配置global_common.conf

[root@master drbd]# cd /usr/local/drbd-8.4/etc/drbd.d/
[root@master drbd.d]# cp global_common.conf global_common.conf.bak
[root@master drbd.d]# vi global_common.conf
global {
usage-count no;
}
common {
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
split-brain "/usr/lib/drbd/notify-split-brain.sh root";
out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
}
startup {
wfc-timeout 30;
degr-wfc-timeout 30;
outdated-wfc-timeout 30;
}
disk {
#磁盘读写速度与同步速率的30%
resync-rate 30M;
on-io-error detach;
fencing resource-only;
}
net {
protocol C;
cram-hmac-alg sha1;
shared-secret "mysql-ha";
csums-alg sha1;
verify-alg crc32c;
}
}

4.创建r0资源

[root@master drbd.d]# vi r0.res
resource r0{
 on master{
 device /dev/drbd0; #逻辑设备的路径
 disk /dev/md2; #物理设备
 address  192.168.2.10:7788;
 meta-disk  internal;
 }
 on slave{
 device /dev/drbd0;
 disk /dev/md2;
 address  192.168.2.11:7788;
 meta-disk  internal;
 }
}

5.建立 drbd resource

1
2
3

[root@master drbd.d]# modprobe drbd
[root@master drbd.d]# drbdadm create-md r0
[root@master drbd.d]# drbdadm up r0

6.设置Primary [在master节点操作]

1	[root@master drbd.d]# drbdadm primary --force r0

7.创建DRBD文件系统 [在Mysql主节点的master上执行]

1 2	[root@master drbd.d]# mkfs.ext4 /dev/drbd0 [root@master drbd.d]# mount /dev/drbd0 /raid10/

8.DRBD同步测试

首先，在主服务器上先将设备卸载，同时将主服务器降为备用服务器：

[root@master drbd]# mkdir -p /raid10/mysql/data
[root@master drbd]# cd /
[root@master /]# umount /dev/drbd0
[root@master /]# drbdadm secondary r0

然后，登录备用服务器，将备用服务器升为主服务器，同时挂载drbd0设备到 /raid10目录：

[root@slave drbd]# drbdadm up r0
[root@slave drbd]# drbdadm primary r0
[root@slave drbd]# mount /dev/drbd0 /raid10/
[root@slave drbd]# cd /raid10/
[root@slave raid10]# ls
lost+found mysql

使用中出现脑裂以及解决办法

[root@slave ~]# cat /proc/drbd
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by root@slave, 2013-12-03 09:50:30 0: cs:StandAlone ro:Primary/Unknown ds:UpToDate/Outdated r-----
ns:0 nr:0 dw:2 dr:1684 al:1 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:4

查看日志

[root@slave /]# tail -f /var/log/messages
Dec  3 11:06:05 slave kernel: block drbd0: helper command: /sbin/drbdadm split-brain minor-0 exit code
127 (0x7f00)
Dec  3 11:06:05 slave kernel: drbd r0: conn( WFReportParams -> Disconnecting )
Dec  3 11:06:05 slave kernel: drbd r0: error receiving ReportState, e: -5 l: 0!
Dec  3 11:06:05 slave kernel: drbd r0: asender terminated
Dec  3 11:06:05 slave kernel: drbd r0: Terminating drbd_a_r0
Dec  3 11:06:05 slave kernel: drbd r0: Connection closed
Dec  3 11:06:05 slave kernel: drbd r0: conn( Disconnecting -> StandAlone )
Dec  3 11:06:05 slave kernel: drbd r0: receiver terminated
Dec  3 11:06:05 slave kernel: drbd r0: Terminating drbd_r_r0
Dec  3 11:06:41 slave kernel: block drbd0: role( Secondary -> Primary )

解决方法：

1.需要将现在的master角色修改为secondary

1
2
3

[root@master ~]# drbdadm secondary r0
#该命令告诉slave，secondary上的数据不正确，以primary上的数据为准。
[root@master ~]# drbdadm -- --discard-my-data connect r0

2.我们还需要在slave上执行下面操作

#这样master就能和slave开始连接上了，并且保证数据不会丢失：
[root@slave ~]# drbdadm connect r0
[root@slave ~]# cat /proc/drbd
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by root@slave, 2013-12-03 09:50:30
 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
 ns:0 nr:4 dw:6 dr:1688 al:1 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[root@master ~]# cat /proc/drbd
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 599f286440bd633d15d5ff985204aff4bccffadd build by root@master, 2013-12-03 09:49:22
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
 ns:4 nr:0 dw:1 dr:1015 al:1 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

安装mysql

1.mastr节点

[root@master Linux]# yum install gcc gcc-c++ autoconf automake ncurses-devel libtool-ltdl-devel* -y
[root@master Linux]# useradd -M -s /sbin/nologin mysql
[root@master Linux]# tar zfvx cmake-2.8.12.tar.gz
[root@master Linux]# cd cmake-2.8.12
[root@master cmake-2.8.12]# ./configure
[root@master cmake-2.8.12]# gmake && make install
[root@master cmake-2.8.12]# cd ..
[root@master Linux]# tar zfxv mysql-5.5.25.tar.gz
[root@master Linux]# cd mysql-5.5.25
[root@master mysql-5.5.25]# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql-5.5
[root@master mysql-5.5.25]# make && make install
[root@master mysql-5.5.25]# cp support-files/my-medium.cnf /raid10/mysql/my.cf
[root@master mysql-5.5.25]# rm -rf /etc/my.cnf
[root@master mysql-5.5.25]# ln -sv /raid10/mysql/my.cf /etc/
[root@master mysql-5.5.25]# cd /usr/local/mysql-5.5/
[root@master mysql-5.5]# chown -R root:mysql .
[root@master mysql-5.5]# chown -R mysql:mysql /raid10/mysql/data/
[root@master mysql-5.5]# ./scripts/mysql_install_db --user=mysql \
--basedir=/usr/local/mysql-5.5/ \
--datadir=/raid10/mysql/data/
[root@master mysql-5.5]# cp support-files/mysql.server /etc/init.d/mysqld
[root@master mysql-5.5]# chmod +x /etc/init.d/mysqld
[root@master mysql-5.5]# chkconfig --add mysqld
[root@master mysql-5.5]# vi /etc/init.d/mysqld
datadir=/raid10/mysql/data

2.savle节点安装

——安装mysql[MySQL主节点的savle节点安装]
[root@slave Linux]# yum install gcc gcc-c++ autoconf automake ncurses-devel libtool-ltdl-devel* -y
[root@slaveLinux]# useradd -M -s /sbin/nologin mysql
[root@slave Linux]# tar zfvx cmake-2.8.12.tar.gz
[root@slave Linux]# cd cmake-2.8.12
[root@slavecmake-2.8.12]# ./configure
[root@slavecmake-2.8.12]# gmake && make install
[root@slavecmake-2.8.12]# cd ..
[root@slave Linux]# tar zfxv mysql-5.5.25.tar.gz
[root@slave Linux]# cd mysql-5.5.25
[root@slave mysql-5.5.25]# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql-5.5
[root@slave mysql-5.5.25]# make && make install
[root@slave mysql-5.5.25]# cd /usr/local/mysql-5.5/
[root@slave mysql-5.5]# chown -R root:mysql .
[root@slave mysql-5.5]# cp support-files/mysql.server /etc/init.d/mysqld
[root@slave mysql-5.5]# vi /etc/init.d/mysqld
datadir=/raid10/mysql/data

MySQL主节点实现高可用

注意：以下操作在MySQL主节点的master、slave节点执行

1.添加hosts主机信息

1
2
3

[root@master Linux]# vi /etc/hosts
192.168.2.10  master
192.168.2.11  slave

2.添加用户和组

1 2	[root@master Linux]# groupadd haclient [root@master Linux]# useradd -g haclient -M -s /sbin/nologin hacluster

3.安装heartbeat

安装相关软件依赖包

[root@master Linux]# yum install libtool automake autoconf \
glib2-devel \
libxml2-devel \
bzip2-devel \
libtool-ltdl-devel \
libxslt-devel \
docbook* -y

安装glue

[root@master Linux]# wget http://hg.linux-ha.org/glue/archive/glue-1.0.9.tar.bz2
[root@master Linux]# tar jfvx glue-1.0.9.tar.bz2
[root@master Linux]# cd Reusable-Cluster-Components-glue--glue-1.0.9/
[root@master Reusable-Cluster-Components-glue--glue-1.0.9]# ./autogen.sh
[root@master Reusable-Cluster-Components-glue--glue-1.0.9]# ./configure LIBS='/lib64/libuuid.so.1'
[root@CentOS Reusable-Cluster-Components-glue--glue-1.0.9]# make && make install

安装agents

[root@master Heartbeat-3-0-7e3a82377fa8]# cd ..
[root@master Linux]# wget https://codeload.github.com/ClusterLabs/resource-agents/legacy.tar.gz/v3.9.2
[root@master Linux]# tar zfvx ClusterLabs-resource-agents-v3.9.2-0-ge261943.tar.gz
[root@master Linux]# cd ClusterLabs-resource-agents-b735277/
[root@master ClusterLabs-resource-agents-b735277]# ./autogen.sh
[root@master ClusterLabs-resource-agents-b735277]# ./configure LIBS='/lib64/libuuid.so.1'
[root@master ClusterLabs-resource-agents-b735277]# make && make install

安装heartbeat

[root@master ClusterLabs-resource-agents-b735277]# cd ..
[root@master Linux]# wget http://hg.linux-ha.org/heartbeat-STABLE_3_0/archive/7e3a82377fa8.tar.bz2
[root@master Linux]# tar jfvx heartbeat-3.0.5.tar.bz2
[root@master Linux]# cd Heartbeat-3-0-7e3a82377fa8/
[root@master Heartbeat-3-0-7e3a82377fa8]# ./bootstrap
[root@master Heartbeat-3-0-7e3a82377fa8]# ./ConfigureMe configure LIBS='/lib64/libuuid.so.1'
[root@master Heartbeat-3-0-7e3a82377fa8]# make && make install

4.配置heartbeat

[root@master Heartbeat-3-0-7e3a82377fa8]# cd doc
[root@master doc]# cp authkeys haresources ha.cf /etc/ha.d/
[root@master doc]# cp /usr/etc/ha.d/shellfuncs /etc/ha.d/
[root@master doc]# rm -rf /usr/etc/ha.d
[root@master doc]# ln -sv /etc/ha.d /usr/etc
[root@master doc]# vi /etc/ha.d/ha.cf
#开启日志
logfile /var/log/ha-log
#设置syslog()/logger设备
logfacility local0
#心跳发送时间间隔/秒
keepalive 2
#15秒没有收到主机心跳、确认主机故障
deadtime 15
#警告次数
warntime 5
#守护进程启动30后 启动服务资源
initdead 30
#监听端口
udpport 694
#另一个节点IP、通过检测来保证心跳的可用性
ucast eth0 192.168.1.11
ucast eth1 192.168.2.11
#两个节点的名字 [uname -n 获取]
node master
node slave
#开启DPOD
respawn hacluster /usr/lib64/heartbeat/ipfail
respawn hacluster /usr/lib64/heartbeat/dopd
apiauth ipfail gid=haclient uid=hacluster
apiauth dopd gid=haclient uid=hacluster
[root@master doc]# vi /etc/ha.d/authkeys
auth 1
1 sha1 HI!
[root@master doc]# chmod 600 /etc/ha.d/authkeys
[root@master doc]# vi /etc/ha.d/haresources
master drbddisk::r0 Filesystem::/dev/drbd0::/raid10::ext4 mysqld IPaddr::192.168.1.254/24/eth0
[root@master doc]# cd /Linux/drbd-8.4.4/scripts/
[root@master scripts]# cp drbddisk /etc/ha.d/resource.d/
[root@master scripts]# cp /etc/init.d/mysqld /etc/ha.d/resource.d/
[root@master scripts]# chkconfig --add heartbeat
[root@master scripts]# chkconfig heartbeat on

测试

1.启动master节点启动heartbeat

1	[root@master scripts]# /etc/init.d/heartbeat start

2.启动slave节点启动heartbeat

1	[root@slave scripts]# /etc/init.d/heartbeat start

3.在master节点上查看启动日志

[root@master ha.d]# tail -f /var/log/ha-log
Apr 26 21:11:48 master heartbeat: [42033]: info: Starting child client "/usr/lib64/heartbeat/ipfail" (501,501)
Apr 26 21:11:48 master heartbeat: [42033]: info: Starting child client "/usr/lib64/heartbeat/dopd" (501,501)
Apr 26 21:11:48 mster heartbeat: [42062]: info: Starting "/usr/lib64/heartbeat/ipfail" as uid 501  gid 501 (pid 42062)
Apr 26 21:11:48 mster heartbeat: [42064]: info: Starting "/usr/lib64/heartbeat/dopd" as uid 501  gid 501 (pid 42064)
Apr 26 21:11:48 mster heartbeat: [42061]: info: Local Resource acquisition completed.
Apr 26 21:11:48 mster heartbeat: [42033]: info: Initial resource acquisition complete (req_our_resources)
Apr 26 21:11:48 mster ipfail: [42062]: ERROR: auto_failback set to incompatible legacy option.
Apr 26 21:11:48 mster heartbeat: [42033]: WARN: Managed /usr/lib64/heartbeat/ipfail process 42062 exited with return code 100.
Apr 26 21:11:48 mster heartbeat: [42033]: info: Status update for slave: status active
harc[42104]:  2014/04/26_21:11:48 info: Running /usr/etc/ha.d//rc.d/status status
#说明启动成功

4.切换主备

在master节点停掉heartbeat

1	[root@master scripts]# /etc/init.d/heartbeat stop

查看slave日志是否能自动切换至slave节点

[root@slave scripts]# tail -f /var/log/ha-log
#显示master节点已经shutdown
Apr 26 21:21:27 slave heartbeat: [42054]: info: Received shutdown notice from 'master'.
Apr 26 21:21:27 slave heartbeat: [42054]: info: Resources being acquired from master.
harc[42118]:  2014/04/26_21:21:27 info: Running /usr/etc/ha.d//rc.d/status status
Apr 26 21:21:27 slave heartbeat: [42119]: info: No local resources [/usr/share/heartbeat/ResourceManager listkeys slave] to acquire.
mach_down[42148]:  2014/04/26_21:21:27 info: Taking over resource group drbddisk::r0
ResourceManager[42175]: 2014/04/26_21:21:27 info: Acquiring resource group: master drbddisk::r0 Filesystem::/dev/drbd0::/raid10::ext4 mysqld IPaddr::192.168.1.254/24/eth0
ResourceManager[42175]: 2014/04/26_21:21:27 info: Running /etc/ha.d/resource.d/drbddisk r0 start
Filesystem[42239]:  2014/04/26_21:21:27 INFO: Resource is stopped
ResourceManager[42175]: 2014/04/26_21:21:27 info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /raid10 ext4 start
Filesystem[42320]:  2014/04/26_21:21:27 INFO: Running start for /dev/drbd0 on /raid10
Filesystem[42312]:  2014/04/26_21:21:28 INFO: Success
ResourceManager[42175]: 2014/04/26_21:21:28 info: Running /etc/ha.d/resource.d/mysqld start
Apr 26 21:21:39 slave heartbeat: [42054]: WARN: node master: is dead
Apr 26 21:21:39 slave heartbeat: [42054]: info: Dead node master gave up resources.
Apr 26 21:21:39 slave heartbeat: [42054]: info: Resources being acquired from master.
Apr 26 21:21:39 slave heartbeat: [42054]: info: Link master:eth0 dead.
Apr 26 21:21:39 slave heartbeat: [42054]: info: Link master:eth1 dead.
Apr 26 21:21:41 slave heartbeat: [42614]: info: No local resources [/usr/share/heartbeat/ResourceManager listkeys slave] to acquire.
Apr 26 21:21:41 slave heartbeat: [42054]: info: Initial resource acquisition complete (req_our_resources)
IPaddr[42642]:  2014/04/26_21:21:42 INFO: Resource is stopped
#显示slave节点的VIP已经运行
ResourceManager[42175]: 2014/04/26_21:21:42 info: Running /etc/ha.d/resource.d/IPaddr 192.168.1.254/24/eth0 start
IPaddr[42727]:  2014/04/26_21:21:42 INFO: Using calculated netmask for 192.168.1.254: 255.255.255.0
IPaddr[42727]:  2014/04/26_21:21:42 INFO: eval ifconfig eth0:0 192.168.1.254 netmask 255.255.255.0 broadcast 192.168.1.255
IPaddr[42701]:  2014/04/26_21:21:42 INFO: Success
mach_down[42148]:  2014/04/26_21:21:42 info: mach_down takeover complete for node master.
harc[42820]:  2014/04/26_21:21:42 info: Running /usr/etc/ha.d//rc.d/status status
mach_down[42837]:  2014/04/26_21:21:42 info: Taking over resource group drbddisk::r0
ResourceManager[42864]: 2014/04/26_21:21:43 info: Acquiring resource group: master drbddisk::r0 Filesystem::/dev/drbd0::/raid10::ext4 mysqld IPaddr::192.168.1.254/24/eth0
#显示slave的mysql和drbd服务已经启动成功
Filesystem[42906]:  2014/04/26_21:21:43 INFO: Running OK
IPaddr[42987]:  2014/04/26_21:21:43 INFO: Running OK
mach_down[42837]:  2014/04/26_21:21:43 info: mach_down takeover complete for node master.

创建RAID1+0组合[存储数据 四块硬盘]

1.创建两块RAID1 [4块硬盘、MySQL主从节点执行]

2.利用两个 RAID1 创建 RAID0

3.将raid信息写入配置文件

安装DRBD

1.解压并安装DRBD

2.安装drbd模块

3.配置global_common.conf

4.创建r0资源

5.建立 drbd resource

6.设置Primary [在master节点操作]

7.创建DRBD文件系统 [在Mysql主节点的master上执行]

8.DRBD同步测试

首先，在主服务器上先将设备卸载，同时将主服务器降为备用服务器：

然后，登录备用服务器，将备用服务器升为主服务器，同时挂载drbd0设备到 /raid10目录：

使用中出现脑裂以及解决办法

查看日志

解决方法：

1.需要将现在的master角色修改为secondary

2.我们还需要在slave上执行下面操作

安装mysql

1.mastr节点

2.savle节点安装

MySQL主节点实现高可用

1.添加hosts主机信息

2.添加用户和组

3.安装heartbeat

安装相关软件依赖包

安装glue

安装agents

安装heartbeat

4.配置heartbeat

测试

1.启动master节点启动heartbeat

2.启动slave节点启动heartbeat

3.在master节点上查看启动日志

4.切换主备

在master节点停掉heartbeat

查看slave日志是否能自动切换至slave节点

创建RAID1+0组合[存储数据四块硬盘]